The Identification and Interpretation of Group Differences on the Canadian Language Benchmarks Assessment Reading Items

This study was undertaken to test the hypothesis that some of the items included in the Canadian Language Benchmarks Assessment Reading Subtest favour certain cultural groups. For example, it was posited that Mandarin speakers, who have a tendency to use bottom-up, local reading strategies, would perform better on particular questions than Arabic speakers, who tend to use topdown, global reading strategies. Two samples of examinees were drawn from previously administered CLBA Form 1, Stage II Reading Assessments. One sample consisted of 250 Arabic speaking immigrants, and the other consisted of 250 Mandarin speaking immigrants. Two ESL reading experts classified each of the 32 CLBA reading items into one of five bottom-up or five top-down reading strategy categories. Differential bundle functioning analyses were conducted to determine whether groups of CLBA items, classified according to the bottom-up, top-down organizing principle, functioned differentially for equal ability Arabic and Mandarin ESL learners. Systematic group differences were found in two of the bottom-up strategy categories and three of the top-down categories. Items involving breaking lexical items into smaller parts and matching key vocabulary in the text to key vocabulary in the item were found to favour the Mandarin speaking examinees, whereas items involving skimming for gist, connecting or relating information presented in different parts of the text, and predicting what may happen in a related scenario were found to favour the Arabic speaking examinees. In sum, five of the ten hypotheses based on the reading strategy framework were supported by the bundle analyses. Therefore, to some extent, this study provides evidence for the validity of the bottom-up, top-down reading strategy organizing principle.